Skip to content

Wip/modeling commons/patch in place - #61

Open
omargfh wants to merge 17 commits into
app/modeling-commonsfrom
wip/modeling-commons/patch-in-place
Open

Wip/modeling commons/patch in place#61
omargfh wants to merge 17 commits into
app/modeling-commonsfrom
wip/modeling-commons/patch-in-place

Conversation

@omargfh

@omargfh omargfh commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

omargfh added 17 commits August 10, 2026 13:49
- Move file-key building, .nlogo parsing, legacy row access, dedupe and the
  per-node migration routine out of archive.ts so patch.ts can reuse them.
- Cover the extracted code with unit tests, including the exact write sequence
  createModelFromNode emits against a recording transaction.
- Fix parseNlogox: it called DOMParser, which Node does not have, so every
  .nlogox model silently stored a null netlogoVersion and infoTab. Replaced
  with a dependency-free parser. Already-migrated rows still hold nulls.
- Pin dob to UTC midnight. pg reads a `date` as local midnight while Prisma
  stores @db.Date from the UTC day, so a birthdate never compared equal to
  itself and shifted by a day in positive UTC offsets.
- Replace the shell-based avatar copy with fs.copyFile; it interpolated a
  legacy filename into a command string.
- Type-check prisma/lib and the migration scripts.
archive.ts only skips rows that already carry a legacyId, so it cannot see
edits, deletions or new children of an already-migrated node. patch.ts applies
those from a diffdb.sh diff.

- Materialises the whole change set as a plan before writing, so the dry run
  and the real run compute the same thing. Applying replays it in one
  transaction and writes an expectations manifest for --verify-only.
- Trusts only `side` and `id` from the diff and re-reads rows from the legacy
  snapshot, since diffdb.sh md5s the `contents` columns. Deleted rows are the
  exception; their row_to_json is the only surviving copy.
- Addresses everything by legacyId. No existing row's id column is touched.
- Recomputes previews from the whole attachment set rather than patching them:
  a preview is one column on the latest version and the highest legacy
  attachment id wins it.
- Moves tags and the preview onto an appended version, matching archive.ts,
  which keeps them on the latest version only.
- Leaves updatedAt alone; Prisma's @updatedat overrode it on create, so no
  migrated row ever held the legacy value.
- Refuses to guess on mid-sequence version deletes, out-of-order appends and
  in-place attachment edits, none of which can be mapped back safely.
Builds a legacy database with a migrated baseline and an incoming schema
carrying sixteen change scenarios, archives the baseline, diffs, patches, then
archives the incoming snapshot into a second database and compares canonical
uuid-free dumps of both.

Asserts that archive(baseline) + patch(diff) equals archive(new snapshot),
with the soft-deleted model checked separately since a fresh archive never
sees it.
A node that gained a version and lost its last preview attachment in the same
diff kept the old preview: the resync compared against the database row for the
appended version, which does not exist yet, read null, and concluded a cleared
preview was already applied. The append then carried the dead key forward.

Compare against what the append will carry instead, and clear that carry when a
resync supersedes it. Covered by a rehearsal scenario; the script's own verify
passed this because the manifest was built from the same wrong plan.
From a review pass over the patch script.

- Derive a version's object key from its legacy id instead of a fresh uuid, and
  diff the row before writing. A modified version previously uploaded a
  duplicate object and repointed netlogoFileKey on every run, so re-applying
  the same diff was not the no-op it was documented to be.
- Refuse an ambiguous ModelAdditionalFile match rather than deleting whichever
  row came back first. Two attachments on one model sharing a filename, or a
  null created_at, could otherwise delete the wrong file irrecoverably.
- Drop a tag from the resolution map once it is scheduled for deletion, so a
  tagging in the same diff is not created and then cascade-deleted.
- Assert the appended version's preview in the manifest; only carried tags were
  covered, so a wrong carried preview could pass verification.
- Strip every uuid segment when comparing preview keys, not just the first,
  which was the model uuid and identical on both sides.

Rehearsal now covers an in-place version edit.
…gacy-migration

- rename archive.ts, patch.ts and archive-upload.ts to initial-import.ts,
  apply-diff.ts and upload-files.ts, naming the job rather than the source
- move the staged output and avatar snapshot under the same directory
- run db:legacy:patch through tsx directly; wrapping it in `sh -c` swallowed
  the --apply / --verify-only / --skip-upload flags it reads from argv
- ModelAuthor.collaboratorType carries legacy collaborator_types.name
- NonMemberContributor archives credited people who never held an account

Both are provenance only; no business logic reads either.
initial-import.ts derived ModelAuthor from version history alone, so a
collaborator who never uploaded a version had no row at all.

- members fold into ModelAuthor as role=contributor, which grants write
  access since canWrite treats any contributor as a writer
- non-members are archived in NonMemberContributor, unread by the app
- idempotent: keyed on the existing (modelId, userId) rows and on
  NonMemberContributor.legacyId, so a second run reports nothing to do
migrateInteractions tracked its per-table count locally and logged it, but
never wrote it back to the report, so report.json always claimed zero. The
docs took that at face value: the phase writes ~6.2M rows on the production
snapshot.

Also records the full-scale rehearsal and the recompute step, which nothing
referenced even though the import leaves the denormalized Model counters at 0.
sessions (1.3M rows) and ip_locations (120k) map to nothing in the new schema,
so apply-diff.ts always discarded their diffs. Scanning them was most of the
script's runtime. Output is byte-identical without them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant